Learning DNF by Approximating Inclusion-Exclusion Formulae

Probably Approximately Correct learning algorithms generalize a small number of examples about an unknown concept into a function that can predict a future observation. More formally, let X and Y be the instance and outcome spaces, respectively. Then a PAC algorithm observes randomly drawn examples (x; f(x)) about an unknown concept f : X ! Y . These examples are independently and identically distributed random variables governed by an arbitrary and unknown distribution over X . With and only with these training examples, the algorithm aims to nd a hypothesis h : X ! Y that approximates the target concept f with respect to the same distribution. Hence it measures \goodness" of the hypothesis h by the probability acc(h) = Probx2Xfh(x) = f(x)g called the prediction accuracy. Valiant introduced the PAC model in a series of papers [12, 13], which is currently one of the most standard platforms for invention of polynomial-time learning algorithms. The PAC theory aims to learn as much general concept classes as possible, beginning from simple structures, e.g. depth-one or depth-two Boolean circuits. Valiant proved that Boolean conjunctions are polynomial-time learnable, and left the learning problem of the class DNF = fpolynomial-size Disjunctive Normal Form formulaeg for the future research. Here, as usual, a DNF formula is a disjunction of a amily of conjunctions of Boolean literals. These Boolean conjunctions are commonly called the terms of the DNF formula. The size of a DNF formula is the number of its (distinct) terms. Since then, a lot of literatures have proved learnability of subclasses of DNF by specifying either structural parameters of formulae or the distribution for the training examples ([1] provides a list of literatures).